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DARPA  Revolutionizing  Prosthetics  2009  Program 

■  Vision 

■  Produce  a  fully  neurally  integrated  upper  extremity  prosthetic  with  appropriate 
documentation  for  clinical  trials,  FDA  approvals,  ana  manufacturing  transition. 


■  Mission 

■  Apply  an  understanding  of  the  underlying  function  and  control  of  the  human  arm 
ana  hand  when  performing  the  basic  functions  of  reaching,  pointing,  grasping, 
and  coordinated  finger  movements  to  the  design  of  this  prosthetic. 

■  Understand  and  address  the  amputees’  needs  to  promote  and  enhance  quality  of 
life  issues  -  comfort,  cosmesis,  natural  control,  integrated  sensory  feedback 


This  vision  enabled  APL  to  rapidly  assemble  a  world-class  team  of  neural  scientists , 
clinicians,  technology  developers,  and  commercial  organizations  for  the 

Revolutionizing  Prosthetics  program 
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for  our  injured  warfighters 


Modular  to  suit  range  of  Upper  Extremity  Patients 
Suitable  for  Range  of  Injury  Levels 
(Modular  Mechanical  Design) 

Dexterous  control  of  22+  degrees  of  freedom 
-  mimics  natural  limb 
Natural  Control 
Natural  Performance 

Anthropomorphic,  speed,  dexterity,  force 
Supports  Activities  of  Daily  Living 
Sensory  Perception  of  Environment 
Pressure,  Force 
Temperature 
Tactile  Discrimination 
Proprioception 
Natural  Appearance 
Comfortable 
Durable,  Reliable 
Provides  Suitable  Function  at  Varying  Degrees  of  Invasiveness 
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APL 


Revolutionizing  Prosthetics  2009 
Collaborative  Partners 
>  30  Participating  organizations; 

4  year,  $70+ M  program 


•  Program  Management 

•  System  Engineering  &  Integration 

•  Signal  &  Image  processing 

•  Virtual  Reality  &  Controls 


JHMI 

•  PI  for  Medicine 

•  Human  Subjects 

•  Brain  Computer  Interface 


new  world 


New  World  Assoc 

•  Mechanical  Design 

•  Prototype  Development 

•  Test  &  Integration 


UCI 

•  Sensory  Integration 

Otto  Bock 


rtJJ  '  Prosthet'c  Components 

CZ44<y  Tfoofe.  .  Control  Bus  Architecture 
•  Clinical  Support 


klnea  cfcr 

innovations  in  fiuman-inleractive  motion  control 


Kinea 

•  Mechanical  design 

•  Controls 


Duke 


Duke 

•  Intelligent  Grasp  Control 

•  Sensory  Stimulation 


Northwestern 

•  Non/Low  Invasive  Strategies 

•  Enhanced  Haptics 

•  Mechanical  Design 


RIC 

•  Virtual  Reality  Environment 

•  Targeted  Reinnervation 

•  Patient  Needs 


CALTECH  CalTech 


•  Higher  Cortex 
Signal/Intent  Extraction 


OrthoCare 

I  n  novations 


Orthocare 

•  SMART  socket 

•  Haptics  Patient  Interface 


use 

UNIVERSITY 
OF  SOUTHERN 
CALIFORNIA 


use 


•  Virtual  Reality 
Environment 


U  of  Rochester 

•  Cortical  Control  for 
Hand 

Movement 


Rutgers 


Rutgers  Univ 

•  Peripheral  Nerve  Interfaces 

•  Bio-Materials 


wi  imi 


Stanford  Univ 

•  Cortical  Signal  Extraction 

*  Signal  Analysis  &  Control 


A 

UNB 

UNIVERSITY  OF 
NEW  BRUNSWICK 


UNB 

•  Neural  Integration 

•  Grasp  coding 


u 


THE 

UNIVERSITY 
of  UTAH 


Utah 

•  Peripheral  Nerve  Arrays 

•  Neural  Integration 

•  Wireless  Electronics 


Siqenics 

•  IMES  Implants 

•  Integrated  Electronics 
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Program  Plan 


RP  2009  Roadmap 


Mechatronic 

Arm 

Development 

Biomimetic 

Arm 

Development 

Haptics 

Development 

Neural  Control 
Development 

Communications 
Power,  Control 


22  DOF,  Haptics 

R 

isk  Mitigation 

r 

Gas  Actuators 

. .  I - 

Limb  Approach 

Virtual  Reality 
Environment 
Functional  & 
Patient  Testing 


Prototype  1- 


Low  Risk,  7  DOF 


Prototype  2- 


Candidate  Haptics  Approaches  ^ 


— - 

Neural  Approach 

Cortical  Interface  (UEA,  others) 

\ 

Wired,  Wireless, ... 

Advisory  Panels,  Human  Subject  Venues,  FDA  Approval,  IRBs 


YEAR  1 


YEAR  2 


Phase  I  -  BASE  PERIOD 


YEAR  3 


YEAR  4 


System  Integ 

;  Test  and  Evaluation 

S  Final  Report  -  Base 

ration  Plan 

Master  Plan 

Period 

Limb  Development,  Integration, 

Testing  &  Transition 

' 

/  Final  Repo 

Final  Limb _ . 

IDE  to  FDA 
rt  -  Option  Period 

gorithm  Mapping,  Motor  Control  Strategies,  Training 

Phase  II  -  OPTION  PERIOD 
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Transition  Plan  Clarification: 
R&D  through  Production 

Advanced  Mechatronic 
Limb  Transition  path 


Revolutionizing  Prosthetics 
Phase  1  Highlights 


Proto  2  Intrinsic 


Proto  2  Extrinsic 


Alfred  E.  Mann  Foundation  RFB 
BION®  Package  has  IDE  FDA 
approval  for  use  in  FES  stimulator 
applications. 
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Revolutionizing  Prosthetics  2009  -  Prototype  1 


Prototype  1  at  RIC  -  January  2007 
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Images  Courtesy  of  RIC  Collaboration 
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-  Phase  I  risk  reduction  path  to  final  limb 
"  Electromechanical  actuation 
-  All  degrees  of  freedom,  speed,  torque 


Platform  for  testing  evolving  neural  control  during  Phase  II 
■  Evaluate  emerging  Control  strategies 
"  Explore  sensory  feedback 
"  Understand  limb  design  trade-offs  for  final  limb 


Prototype  2  Objectives 


-Addressing  DARPA  Requirements  Earl 
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Proto  2  Arm  Architectures 


Shoulder 
2  DOM/F 


Humeral 
.Rotator 
1  DOM/F 


Intrinsically  Actuated  Hand 

21  DOM  /  26  DOF 


Extrinsic 
Hand 
11  DOM 
21  DOF 


Intrinsic 
Hand 
15  DOM 
19  DOF 


Intrinsic  hand 
contains  motors 
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Cobot 
drives  hand,  wrist, 
radial  rotator 


Extrinsically  Actuated  Hand 

18  DOM/ 26  DOF 


Elbow 
1  DOM/F 


Wrist 
3  DOM/F 


ii 


Proto  2  Upper  Arm  Performance 


Shoulder 

■  Flex  /  Extend 

■  Abduct  /  Adduct 

■  Humeral  Rotation 

■  45  ft-lbf,  1 20°/sec 


Elbow 

■  Flex  /  Extend 

■  60  ft-lbf,  120°/sec 


Wrist 

■  Flex  /Extend 

■  Pronate  / 
Supinate 

■  Radial  /  Ulnar 
Deviation 


87* 


Hand  Grasps 
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Proto  2  Hands 

■  Intrinsic 

■  15  motors  in  hand 

■  15  actuated  degrees  of  motion 

■  4  underactuated  degrees  of  motion 

■  19  degrees  of  freedom 

•  Extrinsic 

■  Motors  and  transmission  (cobot)  in  forearm 

■  Cobot:  one  power  motor,  15  steering  motors, 
1 5  outputs 

■  1 1  actuated  motions  with  1 1  tendons 

■  7  underactuated  degrees  of  motion 

■  21  degrees  of  freedom 


14 


Prototype  2  Extrinsic  Hand  Dexterity 
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Prototype  2  Intrinsic  Hand  Dexterity 
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Virtual  Integration  Environment 


Input 


Signal 

Analysis 


*• 


Controls 


>■ 


Plant 


■  Complete  limb  system  simulation  environment 

■  Supports  engineering  development, 

■  Neural  signal  acquisition 

■  Algorithm  development 

■  Mechanical  approach  evaluation 

■  Patient  training  /  therapeutic  applications 

■  System  performance  validation  and  design  compliance 


■  End-to-end  interactive  simulation 

■  Acquires  control  signals  (myoelectric,  mechanical,  neural, 
other) 

■  Signal  Analysis:  Interprets  the  intention 

■  Controls:  Translates  intention  into  movement  of  a  virtual 
limb 

■  Allows  the  user  to  interact  with  objects  with  feedback 
(haptics  or  other) 

■  Modular  and  configurable 

■  Support  various  limb  models  and  control  algorithms 

■  Engineering  test  bed  for  improvement  of  these  designs 

■  Evaluate  patient  interfaces  for  control  signal  extraction  and 
sensory  feedback 


-ini  xl 
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Advanced  Actuation 


22-Feb-10 


Monopropellant 

Pneumatics 


Broader  Power 
Alternatives 
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Tendon  Management  Plate 


Mesofluidics 


Cobotics 


Puck  2 


Ultrasonic  Drive  Board  @  20cc 


Main 
Watt  Motor) 


Hand  Control  Board  @  20cc 


18 


Modular  Prosthetic  Limb  Toolkit 


Modular  Prosthetic  Limb  (MPL)  modules  can  be  assembled  in 
any  combination  for  use  with  different  amputation  levels 


Forearm  with 
batteries 


3  DOM  Wrist 


Elbow 


Hand  with  up  to  14  DOM,  1 8  joints, 
Limb  Controller,  and  Sensor  Matrix 


Up  to  21  motors 

Intrinsically  actuated 
hand 

Lithium  batteries 
Less  than  8  lb 

Approaches  human 
strength 

120  deg/s  arm  motion 
360  deg/s  hand  motion 


2  DOM 
Shoulder 


Humeral 

Rotator 
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Neural  Interface 
System  Block  Diagram 


Rectification  Linear  Filter 


stimulus 
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B  0  DE3/  e-pTJ 

Jerk  ' -  -  - -  Htsis*  “ - ' 
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Post  spike  inhibitory  current 


Sensory 

Encoding 
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Limb  Control 
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Neural  Integration 

Research  Components 

Cal  Tech 

U  of  Utah 

ASU 

use 

URMC 

NUPRL/ 

Sigenics 

Zyvex* 

RIRC*/UBN 

JHU 


High  Level  Cortical,  Reach  Decoding  and  Prediction 

Peripheral  Nerve,  Efferent  and  Afferent 

Cortical,  Reach  and  Hand  Positioning 

Simulation  Environment  and  Biomimetic  Control 

Cortical,  Dexterous  Digit  Manipulation 

Wireless  Injectable  EMG  Recording  Methods 

Wireless,  Direct  Peripheral  Nerve  Interface  Methods 

Targeted  Motor  Reinnervation  and  Signal  Analysis 

Signal  Classification,  Synthesis,  Simulation, 
and  Hybrid  Integration 

CLO-05-003  Revl 
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Neural  Interface  Toolkit 
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Conventional  Prosthetic  Control  &  Non-invasive  devices 

■  Surface  EMG  (sEMG) 

■  Tactile  sensory  stimulator  (Tactor) 

Minimally-invasive  devices 

■  Implantable  MyoElectric  Sensor  (IMES) 

Implantable  Peripheral  devices 

■  Utah  Slanted  Electrode  Array  for  Recording  (USEA-R) 

■  Utah  Slanted  Electrode  Array  for  Stimulating  Feedback  (USEA-S) 

■  Next  Generation  Peripheral  Interface  Arrays  using 
Biodegradable  Structures 

Implantable  Cortical  devices 

■  Epidural  electrocorticography  grid  (ECoG) 

■  Floating  Microelectrode  Array  (FMA) 

■  Utah  Electrode  Array  (UEA) 

Multimodal  System  Components  (Multi-BID/Multi-PID) 

■  Multi-Mode  Control  Unit  (MCU) 

■  Headcap 

■  Armband 
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Full  control 

Hand,  Wrist,  End  Effector 


Simultaneous 
Joint  Actuation 


Simultaneous  Finger 
Movements 


Electrode  Caps 


Wrist  Orientation 


End  Effector 
Velocity 


Alfred  E.  Mann  Foundation  RFB 
BI0N®  Package  has  IDE  FDA 
approval  for  use  in  FES  stimulator 
applications. 


End  Effector, 


Hand 


Command 

Conflict 

Resolver 
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Neural  Integration  Strategy  with  Sensory  Feedback 


Perception  and  User  Modified  Intent 


Sensors 


► 


Proprioception 

(position) 


Tactile 

(contact,  pressure,  temp)' 


Embedded  Intelligent  Control  Algorithms 


Peripheral  Afferent  & 
Muscle/Tendon  Stimulation 


Alfred  E.  Mann  Foundation  RFB 
BION®  Package  has  IDE  FDA 
approval  for  use  in  FES  stimulator 
applications. 


Multi-mode  Haptic  Displays 


Dynamic  Socket  Accommodation  _  /jT)\ 

CLQ-05-003  Revl  1  L 
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Comfort  &  Appearance 
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Body  Attachment 

■  Investigating  multiple  volume  accommodating  and 
dynamic  shape  changing  socket  methods 

1 .  Pneumatic  or  air  filled  bladders 

2.  Hydraulic  or  fluid  filled  bladders 

3.  Vacuum  attachment  methods 

4.  Electro-active  Polymers 

5.  Shape  changing  material  structures 

Cosmesis 


■  Exploring  alternative  materials  and  designs  for  reducing 
stress  on  joints 

■  Establishing  metamerism  insensitive  color  formula 

■  Testing  for  sensor  performance  (force,  vibration,  slip,  thermal) 

■  Testing  alternative  mold  designs  to  improve  fabrication 


APL 

Flexsys 

UM 


Vacuum 

OrthoCare 
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Phase  II  System  Build  &  Integration 


Phase  II  Modular  Limb  System 
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Technology  Transition 

The  Upper  Extremity  Prosthetics  market  alone  may  not  not  have 
the  base  to  sustain  the  advanced  limb  system... 


•  “Opportunities  to  scale”  -  leverage  subcomponents  of 
limb/neural  systems  into  additional  applications 

>  Extend  to  lower  extremity  &  related  rehabilitation 
applications 

>  Military  -^IED  defeat,  battlefield  trauma  care/extraction 
robotics 


> 

> 

> 

> 

> 


Homeland  Defense  chem/bio  response,  rad  mitigation 

Medical  ^  peripheral  nerve  repair,  spinal  injury  mitigation 
rehabilitation,  remote  tele-care 

Space  microgravity  mitigation,  robotic  exploration 


Commercial  *  assistive  robotics,  home  care,  etc. 
Other... 
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IED  Robotics 


2/22/201 0 
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Thfli/Ue  you! 


JOHNS  HOPKINS 

UNIVERSITY 


Applied  Physics  Laboratory 


